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The situation regarding a Scavenger, Copydisk, and InitializeDisk for Oak has been unclear. 
This iTienio is iniended to report the current status of those items and to propose a 

resolution of ihe- items. 

Definitions 

InitializeDisk (Pilot) - A program which takes a virgin disk from the manufacturer and 
foimais it into a Valid Pilot Disk containing no Pilot files. 

Valid Pilot Disk - a disk pack which has been formatted with the proper header, label, and 
data blocks. In addition, bad spots have been identified, marked and removed from service. 

Valid Pilot Volume - A Valid Pilot Disk v/hich additionally contains a proper Pilot Volume 
as described in [Iris]<Purcell>PilotVolumeFormaLmemo (attached). 

Copydisk (Alto) - An existing Alto program which makes a bit-for-bit copy of one disk 
pvHck on another. 

Copydisk (Pilot) - A Pilot application client which makes copies of all of the files on the 
source IMiot disk upon the target Pilot disk* l^his differs from a bit-for-bit copy in that 
multiple FlDs for mutable Pilot files must be avoided. 

Movedisk (Pilot) ~ A Pilot application client program which makes a bit-for-bit copy of 
one Pilot disk pack on another. It differs from the Alto Copydisk in that the source pack 
must be erased or otherv/ise made permenanily unavailable so as lu avoid duplicate FlDs for 
the same mutable Pilot file. This should be used only when a pack contains only immutable 
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files or when the physical integrity of the pack itself is suspect (and the user wishes to 
discard the physical pack but not the information on it) 

Scavenger (Alto) - An existing, relatively ill defined (see below) Alto program v/hich takes a 
not-too-badly smashed Alto disk and makes it acceptable to a certain set of Alto subsystenis. 

Scavenger (Pilot File) - A Pilot client program which takes Valid Pilot Disk and leaves it 
containing an undamaged Valid Pilot Volume. 

Scavenger (Pilot Disk) - A program which produces a Valid Pilot Disk from a damage*d 
Pilot disk. It identifies and records bad spots and attempts to relocate the overlaid 
information to other places on the disk. 

Scavenger (Star) - A Pilot client program which repairs damaged Pilot client objects which 
are stored in Pilot files. 

Status 

1) Bob Bowering has been in the hospital and unavailable for consultation. His continued 
availability is uncertain. 

2) Steve Purceti has written a memo (attached) capturing the information required to deal 
with the Pilot disk and required to construct a Pilot file scavenger for it. (There are other 
kinds of scavengers, e.g. a Star document scavenger, which will not be discussed here. See 
above.) 

3) i have talked with Jim Morris (the resident Alto scavenger wizard) at some length about 
the status and functions of the Alto scavenger. 

4) I have determined that the current Alto Copydisk will function very well as the putative 
Pilot MoveDisk. 

5) Steve Purcell has already constructed an IniiializeDisk routine which, with minor 
packaging, can be delivered with Oak to serve to initialize Pilot file disks. It does not deal 
with bad spots. 

Alto Scavenger 

I wish to record here some facts about the Alto Scavenger v/hich T gleaned from my 
conversation with Jim Morris. As the author of the Alto Scavenger he has been sifbjected to 
a wide variety of house calls for problems of one kind or another. 

1) Jim confirmed our impression that far-and-away the most important thing is the 
reconstruction of smached and invalid directories and allocation tables. 

2) There was never a good definition of what the Scavenger would do, of where its duties 
would leave off and a subsystems duties would begin. Ex post facto negotions between the 
Scavenger and the major subsystems have left a lot of anomalies, rough edges, and an endless 
wish list 

3) There are specific error modes which have accounted for the bulk of the problems. These, 
are: 
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a) Bad Spots - The current scavenger cannot deal with bad spots. Mike Overton is 
slowly accumulating unusable disks that have bad spots on them. As a result I am 
placing low priority on dealing with bad spot problems in Oak. 

b) Disk Alignment - This is now less visable as many more people have their own 
Altos and packs are shifted less frequently. Jim had a scheme worked out (the 
details of which he could not recall on the spot) which would detect incipient 
misalignment before it became a real problem. It required co-operation' from the 
drive manufacturer. 

c) Power Supply run-away - Many problems could be attributed ta misbehavior on 
the part of the power supplies, causing bad writing on the disk. (I would catagorize 
our problems with IFS during power failures here) 

d) Processor Overload - The Alto has had problems with the microcode tasks not 
reacting within real time constraints under unusual circumstances. (There is an 
infamous bug which caused every sixth FTP page to be badly writen due to a 
combination a microcode tasks collectively taking too much time.) 

e) The generation of UIDs is poorly done (it*s more like a bug) causing more than 
one file to have that same UID. This complicates life for the Alto Scavenger. Jim 
agrees that Pilot has that problem designed out. The problem has never been 
corrected on the Alto simply because the system is not being maintained (the specific 
problem seems easy to fix). 

Proposal 

1) Steve Purceli is to be directed to specify, document, package, and deliver to Oak alpha test 
an InitializeDisk. In Oak it will not deal with bad spots. 

2) Tiie current Aito Copydisk be used as the Pilot IVIovedisk in Oak. This will be run on an 
Alto. Removing the old pack from circulation will be accomplished by operational 

procedures. No Piloi Copydisk will be provided with Oak. 

3) Thai Sieve Purcefl be directed to specify, document, and deliver a Pilot File Scavenger 
which is restricted to reconstruction of the vfm and vam. Neither a Pilot Disk Scavenger 
nor a Star Scavenger will be delivered with Oak. 
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Filed on: [Iris]<PurceU>PilotVolumeFormatmemo 

This memo describes some aspects of Pilot Volumes. Since the design of Pilot and its 
volumes is still in flux, only a snapshot of the current design and implementation can be 
provided. There is no guarantee that what follows will not change. The general structure is 
probably correct and will remain, but many details will change. 

Pilot scores files in volumes which are physical disks with data conforming to certain 
constraints. Pilot assumes the storage to consist of pages randomly accessed by volume- 
page-numbers which range continuously from zero to the size of the volume in pages. Bad 
Pages will be hidden by the disk driver. A single disk Diablo 31 will have volume pages in 
the range [0..4872). As far as Pilot's volume manager is concerned, a page is an 8 word label 
and 256 words of data. This memo ignores additional page fields such as the 2 word address 
header used by the device drivers/controller. 

Pilot partitions pages into a number of files, each with a unique identifier (universaiio) and a 
type. (The generaiion and registration of types is still fuzzy, but many files can have the 
same type). (The file properties of immutable and temporary may be viewed as contained in 
ihe type, although, they are actually independent fields.) Pilot uses four types for system or 
volume files, present on every volume. Each page in a volume belongs to exactly one file 
and has a label vv^iih iPxS file ID, the file type and the file-page-number, A client file has 
pages \vith consacuiive file-page-numbers from zero, while a system file is numbered by 
voiume-page-numbers. which are not necessarily consecutive. Page labels and page data are 
stored together for safety. For efficiency and redundancy, label informaibn is also stored 
in ihe system files, which can be entirely discarded and i*econstructed from labels if 
damaged. The four system file types are root^ vam, vfm and free. The root file is exactly 
one page with a constant location on the volume, containing IDs (and sometimes page 
numbers) of the other system files and of one client root file. The vam (volume allocation 
map) is a bit map telling which pages are free. The vfm (volume file map) is a B-tree 
v/hich maps (file ID, file-page-number) keys into volume-page-numbers for all client files. 
The free file has blank pages scattered over the volume that are not in use either by clients 
or by Pilot. Bad pages can be thought of as free pages but the Pilot volume manager is not 
aware of them, since the disk driver ensures that there is a good page for every volume- 
page -number. 

The root file is created and accessed by 

LogicaiVoiume.RootAccess: PROCEDURE[volume: Voiume.ID, proc: PROCEDURE[VOlume: 
Logicaivoiume. Handle]]; 

LogicalVolume. Handle: TYPE = POINTER TO LogicaiVolume.Descriptor; 
LogicalVolume.Descriptor: TYPE = RECORD[ 

version: cardinal, 



[lris]<Lynch>'Scavengor.Mofno June 15, 1973 



volumeSize: Volume. PageCount, 

vID: Volumfa.lD, 

vam: File. ID, 

vfm: FiiG.ID, 

free: Fiie.lD, 

vam: Fiie.lD 

LogicalVolume.nullDescriptor: LogicalVolume. Descriptors ... ; 

VolumeRoolAccess is a stylized way to read and lock the root page, access and modify it by 
a client procedure proc, and then write it back to the volume and unlock it. The proc can 
use the handle (a pointer) to read and write the root page which will then be written back to 
the volume. 



The vam(volume allocation map, a bit map) is created and accessed by 

VolAHocMapJnit: PROCEDURECvolume: Volume.lD]; 

VoiAiiocMap.GetBusy: PROCEDURE[volumePage: LogicaiVoiume.PageNumber] returns 
[busy: boolean]; 

VoiAHocMap.SetFree: PROCEDURE[volumePage: LogicaiVoiume.PageNumber]; 

VoiAiiocMap.GetSetBusy: PROCEDURE[volumePage: LogicaiVoiume.PageNumber] returns 
[busy: boolean]; 

—set a page- to busy and return its previous state 

VofAiiocMap.GrabFirstFree: PROCEDURE[volumePage: LogicaiVoiume.PageNumber] returns 
[Logica!Vo!um^&.PageNumber]; 

"•find first free page and set busy (may signal VoiumeJnsufficientSpace) 

The vfmCvoluine file map, a B-tree) is created and accessed by 

VciR^eMapJnit: PRCC£DUR£[voIume: Volume.lD]; 

VciFneMap.GetPageGroup: PROCEDURECfile: Fiis.lD, filePage: Fiie.PageNum^ber] returns 
[FiieJnt^maiPageGraup]; „ - " 

VoiFiieMap.GetNext: PROCEDURE[file: Fiie.lD, fn.ePage: Fiie.PageNumber] returns 
[nextFile: Fiie.lD, nextFilePage: Fiie.PageNumber]; 

—starting and ending with nuil, enumerates tf^e page group boundaries 

VoiFiieMap-lnsertPageGroup: PROCEDURE[file: FiieJD, group: Fiieinternai.PageGroup]; 

VoiFiieMap.DeietePageGroup: PROCEDURE[file: Fiie.lD, group: Fiieinternai.PageGroup]; 

The vfm maps keys (file ID, file-page-niimber) into volume pages, and is abstractly a 
collection of entries (file ID, file-page-numiber, volume-page-number). The procedures for 
accessing it use page groups to encode runs of entries with file-page-numbers in a closed- 
open interval: [..). The null volume page resulting from initialization or deletion signifies 
the absence of a file or a page of a file. Entries have unique keys, and insertions overwrite 
existing entries. Therefore pages with duplicate ID and page number cannot be pointed to 
by the map. A scavenger would have to deal with such (illegal) page pairs before updating 
the vfm. The vfm does not depend on consecutive file-page-numbcrs so that as ii is being 
reconstructed, say by the scavenger, it can contain fragmeuts of files. Client files -with gaps, 
however, are not permitted in a legal Pilot volume. Insertions are most efficient when 
clustered by key (ordered by ID, page number). GetNext is used as an enumerater. AH 
client file pages can be located on the volume in random access fashion by use of the vfm. 



